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SUBJECT: LITERACY NOW READING INTERVENTION PROGRAM: A COHORT 
ANALYSIS OF READING ACHIEVEMENT AT SELECTED HISD CAMPUSES, 
2020-2021 


Since 2007, nineteen HISD campuses have partnered with Literacy Now to offer the Reading 
Intervention program to students in kindergarten to second grade who were at risk of failure. The 
evaluation employed a two-group quasi-experimental design, with an intervention group and a 
control group. The research used two cohorts of students who participated in the program in the 
2015-2016 and the 2016-2017 academic year. Data were drawn from the post-reading 
intervention teacher survey administered between 2015-2016 and 2017-2018 and student 
academic performance on Istation’s Indicator of Progress (ISSIP) Early Reading and STAAR 
grade 3 reading. 


Key findings include: 

e Between 2015 and 2019, 996 kindergarten to second-grade students from eleven HISD 
campuses participated in the Reading Intervention program. 

e A post-program survey found that teachers observed that the program was beneficial to 
students (M = 2.62 out of 3.0), students' grades improved (M = 2.51 out of 3.0), and 
students displayed greater confidence in reading (M = 2.48 out of 3.0). 

e First-grade students who participated in the Reading Intervention program showed a 
higher increase in the mean proficiency reading score compared to first-grade students 
who did not participate in the intervention. 

e The percentage of Els students in the intervention reading at grade level increased by 
56.1 percentage points in cohort 1 and 2.5 percentage points in cohort 2 compared to 
their campus-level peers who did not participate in the intervention (44.5 and 0.0 
percentage points, respectively). 

e The percentage of economically disadvantaged students who participated in the 
intervention reading at grade level increased by 45.7 percentage points in cohort 1 and 
1.6 percentage points in cohort 2 compared to their campus-level peers who did not 
participate in the intervention (28.2 and -3.7 percentage points, respectively). 

e The percentage of Hispanic students in the intervention reading at grade level increased 
by 53.6 percentage points in cohort 1 and 2.7 percentage points in cohort 2 compared to 
their campus-level peers who did not participate in the intervention (23.0 and -2.5 
percentage points, respectively). 

e The percentage of Black students who participated in the intervention reading at grade 
level increased by 42.5 percentage points in cohort 1 and 0 percentage points in cohort 
2 compared to their campus-level peers who did not participate in the intervention (32.3 
and 2.0 percentage points, respectively). 

e The 3-year mean number of students in the intervention who met the Approaches grade- 
level benchmark score fell within 20 percentage points of their campus-level peers on 


STAAR grade 3 reading at Ashford (18%), McGowen (15%), Ross, (11%), Walnut Bend 
(10%), and Piney Point (6%). 
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Abstract 

Since 2007, nineteen Houston Independent School District (HISD) campuses have partnered with Literacy Now to offer the 
Reading Intervention program. The Reading Intervention program provides individualized, small group reading tutorials for 
at-risk students in kindergarten to second grade to create proficient readers by the end of third grade. Literacy Now also 
supports literacy development at home (Parent Engagement Workshops, Book Distribution, Family Literacy Night, and 
monthly newsletters). Between 2015 and 2019, 996 students in kindergarten to second grade from 11 HISD campuses 
participated in the Reading Intervention program. During this period, results from a post-program survey, on a 3.0 scale, 
found that teachers observed that the program was beneficial to students (M = 2.62), grades improved (M =2.51), and students 
displayed greater confidence in reading (M = 2.48). The evaluation employed a two-group quasi-experimental design with an 
intervention group and a control group to examine the effectiveness of the program on two cohorts of students from 2015-2016 
and 2016-2017. The evaluation found that kindergarten and first-grade students in both cohorts who participated in the 
reading intervention program showed significant differences between the pre-and post-test scores on Istation for the average 
proficiency rate in reading immediately after completing the program. Compared to their grade-level peers, first graders, 
English language learners, Black, or economically disadvantaged students who participated in the Reading Intervention 
program showed higher percentage points increase in their reading performance on STAAR grade 3 reading assessment. The 
three-year mean percentage of the students who met the Approaches grade-level benchmark score fell within 15 percentage 
points of their campus-level peers on STAAR grade 3 reading at Piney Point (6%), Walnut Bend (10%), Ross (11%), Bruce 
(15%), and McGowen (15%). 


There is an increasing number of middle school and 
high school students diagnosed with reading deficiencies 
(Scammacca, Roberts, Cho, Williams, Roberts, Vaughn, 
& Carroll, 2016). This has increased the need for access 
to effective interventions for struggling readers 
(Scammacca et al., 2016). For students with below- 
grade-level reading performance to catch up with their 
peers, their rate of growth must be accelerated; simply 
learning at an average rate will only maintain the deficit 
(Vaughn, Denton, & Fletcher, 2010). Intensive reading 
intervention programs are a Salient strategy to ensure 
future student success (Vaughn et al., 2010). 

Early intervention has shown to be the most effective 
approach to literacy (Lovett, Frijters, Wolf, Steinbach, 
Sevcik, & Morris, 2017). Ensuring that all children, 
regardless of their early experiences, can read by the 
third-grade positions them to absorb content in later 
elementary grades and establish a strong foundation for 
secondary school learning (Vaughn et al., 2010). 

Although reading by third grade alone will not ensure 
success, research shows that children who are not 
reading proficiently by third grade are four times more 
likely than proficient readers to drop out of high school 


(Hernandez, 2011; 2012). Students who cannot read are 
more likely to be unemployed and adjudicated, and 
require government support (Connor, Alberto, Compton, 
& O’Connor, 2014). Simply said, being able to read is 
the gateway to achievement in today’s literacy-driven 
society. 


Background 


In 2006, Literacy Now, formerly Making It Better, was 
created to address the growing gap in achievement 
among HISD students in low-income Houston 
communities. Since its inception, Literacy Now has been 
providing opportunities to under-served children in the 
Houston Independent School District (HISD) that 
address the needs of the whole child (academic, social, 
and emotional) and includes their families in the process 
(Literacy Now, 2019). 

The principal theoretical basis for the Literacy Now 
approach is that communities can be transformed by 
empowering children and families through literacy, 
leadership, and life skills (Literacy Now, 2019). The 
primary goal is to serve HISD students, their families, 


schools, and the surrounding communities with 
excellence by offering quality support services to 
enhance students’ social emotional learning and provide 
a foundation for academic success. The Literacy Now 
program aims to create proficient readers by the end of 
third grade (Literacy Now, 2019). 


Program Elements 

The Literacy Now flagship program is Reading 
Intervention, which provides individualized, small 
group reading tutorials for at-risk students in 
kindergarten to second grade. The Reading Intervention 
program consisted of 45-minute sessions delivered 
during the school day 2 or 4 times a week, with a trained 
reading interventionist. Students were leveled and 
grouped according to a comprehensive pre-assessment. 

The Reading Intervention program was tailored to 
meet the individual needs of students using a pre- 
assessment. The pre-assessment provided a baseline 
measure of student progress and was used to build 
lessons. Mid-year results, in conjunction with ongoing 
monitoring of student progress, were used to drive lesson 
planning and develop instructional strategies. Post- 
testing in April measured students’ progress and 
program success (Literacy Now, 2020). Ongoing 
evaluations were used to create lessons for 
individualized learning, as well as strategically group 
students to accelerate progress. Meetings were held with 
teachers and school administration to discuss progress 
and identified additional assistance that may be needed 
for specific students (Literacy Now, 2019). 

The program operated on a low student to tutor ratio, 
maintaining a 4:1 (or lower) student to intervention ratio 
(Literacy Now, 2019). This low ratio provided students 
with individualized interventions and _ allowed 
interventionists to develop a meaningful relationship 
with each child. In terms of cost-effectiveness, the 
portion that a school paid towards the program depended 
upon the needs and funds available at each campus, the 
number of students to be served, and the length of the 
intervention. The program typically consisted of 25 
weeks of instruction and served 64 students at a cost of 
approximately two thousand dollars per student 
(Literacy Now, 2019). 

As part of the Reading Intervention program, the 


Literacy Now Interventionists are trained on the 
Neuhaus Reading Readiness program curriculum. The 
Neuhaus Reading Readiness program is based on the 
five components of reading as determined by the 
National Reading Panel: (i) Phonemic Awareness; (ii) 
Phonics; (iii) Fluency; (iv) Comprehension; and (v) 
Vocabulary (Literacy Now, 2019). 
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The Neuhaus Reading Readiness program 
curriculum is designed for students to learn the 
fundamentals of reading through multisensory activities. 
Through the curriculum, interventionists work with 
students on the five components of reading, which are 
used concurrently in daily lesson planning and 
supplemental materials (Program Administrator, 
personal communication, January 7, 2021). Activities 
included instant letter recognition skills, phonological 
awareness, oral language skills, sound-symbol 
correspondences, and handwriting. 

The Reading Intervention program goes beyond the 
classroom by supporting literacy development at home 
through Parent Partnership programs and monthly 
newsletters that encourage literacy activities at home 
(Literacy Now, 2019). Students choose age-appropriate 
books throughout the program to help build their home 
libraries and receive books and materials at the end of 
the program to help reduce summer slide - the decline in 
reading ability that can occur over the summer months. 
Two workshops were held where parents met with 
interventionists and received materials, support, and 
additional skills to foster their child’s literacy 
development and expand learning at home (Literacy 
Now, 2019). 

In conjunction with the Reading Intervention program, 
as part of Literacy Now, HISD campuses were provided 
additional support and outreach activities. The Parent 
Engagement Workshops provide a space for parents to 
connect while participating in fun, free, hands-on 
activities that could also be used at home (Literacy Now, 
2019). Parents also learned how their children could get 
the most benefit from the books they received each 
week. GROW (now Book Distribution) provided age- 
appropriate and culturally sensitive books to children in 
the Reading Intervention program as well as other 
programs (Literacy Now, 2019). 

Several outreach events were incorporated into the 
Literacy Now model, geared towards fostering a 
community of learning. Family Literacy Night was a 
free, fun, and interactive outreach event that provided 
materials, education, and activities to encourage reading 
as a way of life (Literacy Now, 2019). Each Family 
Literacy Night included numerous literacy stations 
staffed by community volunteers and were geared 
towards engaging both parent and child(ren) in literacy- 
rich activities. Read Aloud Month is a national campaign 
focused on sharing the love of reading (Feb- 
Mar) (Literacy Now, 2019). Volunteers contributed to 
closing this gap by reading aloud for 30 minutes/week (3 
times total) in a first grade or second grade classroom in 
one of the program schools. Finally, Literacy 
Enrichment Events provide an _ opportunity for 


companies and community friends to serve together 
through 45-minute reading enrichment activities. They 
also provide Reading Intervention program students 
(kindergarten to second grade) with a fun learning 
opportunity (Literacy Now, 2019). 


Literature Review 


There is a need for policymakers and educators to 
focus on closing the achievement gap between 
disenfranchised student populations and _ their 
counterparts (Everson & Bussey, 2013). Research has 
shown that early intervention for students at-risk of 
failing due to their inability to read at grade level has 
allowed students to make reading gains that were 
sustained after the intervention (Lovett, Frijters, Wolf, 
Steinbach, Sevcik, & Morris, 2017). 

There are various delivery arrangements for school- 
based reading interventions for struggling readers. 
Stentiford, Koutsouris, & Norwich (2018) conducted a 
systemic literature review (1970-2017) and found that 
interventions targeting students who did not respond 
well to whole-class instruction were almost exclusively 
delivered in pull-out sessions by people other than the 
classroom teacher. This practice was found to be 
prevalent in primary schools and suggests that students 
that were identified for pull-out tutoring supports have 
less access to their teacher’s time and expertise 
(Stentiford, Koutsouris, & Norwich, 2018). 

In looking at group instruction rather than one-on- 
one tutoring, Harpine & Pazda (2019) found that 
students in the group-centered intervention group who 
were subject to hands-on teaching/counseling approach 
showed greater improvements than those in one-on-one 
tutoring. Those in the group-based tutoring continued to 
show growth 6-weeks after the 1-week intervention. 

Researchers have found that interventions of small 
groups of three to five students might be more effective 
in addressing reading difficulties than those in groups of 
six or more students (e.g., Baker & Kosty, 2012). 
Providing reading interventions to at-risk students in 
small groups increases students’ participation, teachers’ 
monitoring, and provides appropriate response to 
students’ progress (Vaughn, Wexler, Leroux, Roberts, 
Denton, Barth, & Fletcher, 2012). When at-risk students, 
especially younger ones, receive interventions in small 
groups, and are provided extended opportunities for 
practice and daily feedback from teachers, they are better 
able to acquire the foundational skills needed for reading 
(Vaughn, Wanzek, Wexler, Barth, Cirino, Fletcher & 
Francis, 2010). 

Early intervention is key to reading success (Lovett, 
Frijters, Wolf, Steinbach, Sevcik, & Morris 2017; 
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Vaughn et al, 2010). A study comparing the progress of 
kindergartners who received small group reading 
intervention for the full school year with those beginning 
the same intervention midyear found that explicit, 
systematic reading instruction provided early in the 
kindergarten year is more beneficial to students 
identified as at risk for reading failure than delaying 
reading intervention until midyear (Helf, 2017). 

Enlisting parents to help teach reading had been 
found to be beneficial as well. Parents can assist with 
literacy instruction, providing critical pre-reading skills 
at home that increase vocabulary and performance on 
pre- reading skills sub-test in schools (McConnell & 
Kubina, 2016). Also looking at parental engagement, 
Jones and Christian (2020) evaluated SPARK; a 
Milwaukee based reading intervention program 
delivered by a community organization. The SPARK 
program paired one-on-one tutoring with parent 
engagement and was delivered over two years to low 
income and low performing Milwaukee Public Schools. 
The research found that there was a significant impact 
on participation, literacy, and school attendance. The 
greatest impact was on those students with the greatest 
need for additional reading support. 

When parents engage in a partnership with the 
educational environment, they acquire tools that help 
them increase their knowledge on how best to support 
their children (Pong & Landale, 2012). In a study of 
academic performance of disenfranchised groups in U.S. 
schools, Carnoy and Garcia (2017) used individual 
student microdata gathered from NAEP over 10 to 17 
years to estimate changes in race/ethnic gaps, controlling 
for English-language learner designation, gender, and 
socioeconomic status. The research found that English 
learners (ELs) were falling further behind white students 
in mathematics and reading achievement. 

Additionally, it was found that the gaps between 
higher- and lower-income students persisted, with some 
changes that varied by subject and grade (Carnoy & 
Garcia, 2017). Meanwhile, the proportion of low- 
income students in U.S. schools had rapidly increased, 
as had the share of minority students in the student 
population. The chances of ending up in a high-poverty 
or high-minority school were highly determined by a 
student’s race/ethnicity and social class. For example, 
black and Hispanic students—even if they were not 
poor—were much more likely than white or Asian 
students to be in high-poverty schools (Carnoy & Garcia, 
2017). 

Differences in student’s socio-economic, 
demographic, and educational background explained 
approximately 15% of the variation in low performance 
across students (Organization for Economic Co- 


operation and Development [OECD], 2016). While 
social and demographic background do not determine 
student achievement, they do create the conditions for 
opportunities — or the lack of them — that influence 
students’ progression through the school system 
(OECD, 2016). Early intervention is the key to 
mitigating these risk factors (Carnoy & Garcie, 2017). 


Research Questions 


Due to the importance of reading on students’ future 
academic success, the Literacy Now program was 
implemented at several HISD campuses. This evaluation 
examines the types of support offered to parents, the 
effect the Reading Intervention program had on student 
performance, and how these outcomes vary based on 
demographic characteristics. As such, the research 
questions are as follows: 


1. What was the demographic and _ educational 
composition of students in the Literacy Now 
program between 2015-2016 and 2018-2019? 


2. What Literacy Now support services were provided 
to HISD campuses from the 2015-2016 to the 2018- 
2019 academic year? 


3. What were teachers’ observations of students’ 
progress in the Reading Intervention program from 
the 2015-2016 to the 2018-2019 academic year? 


4. How did the performance of kindergarten to second 
grade students in the Reading Intervention program 
compare with their campus grade level peers on the 
Istation reading assessment? 


5. What were the differences in reading performance 
between the students in the Reading Intervention 
program and their peers when demographic 
characteristics are taken into consideration? 


6. What was the longitudinal effect of the Literacy 
Now program on the STAAR Reading performance 
of HISD third grade students? 


7. How did students in the Reading Intervention 
program compare with their campus grade level 
peers on the STAAR grade 3 reading assessment 
when demographics are taken into consideration? 


Based on the objective of the Literacy Now program, 
that students should be reading by grade three, this 
evaluation is predicated on the assumption that students 
who received the intervention would show increased 
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performance on measures of reading immediately 
following the intervention. Furthermore, it would be 
expected that those gains would remain through to third 
grade following participation in the program. Finally, it 
was hypothesized that Literacy Now would also promote 
reading skills among subgroups of disenfranchised 
students identified as limited English proficiency, Black, 
Hispanic, or economically disadvantaged. 


Method 


This evaluation used data on kindergarten to second 
grade students in one of the eleven schools that 
participated in the Literacy Now program between 
2015-2016 and 2018-2019. Principals selected students 
for the Literacy Now program based on their reading 
performance on district-level assessments, standardized 
assessments, and principal / teacher recommendations. 

Starting in October of each academic year, selected 
students would be provided with tutoring two to four 
times a week over 25 weeks using a pull-out model of 
instruction. Through various supplemental programs 
(Parent Engagement Workshops, Book Distribution, and 
Family Literacy Night), parents were provided with 
strategies to promote reading at home. The evaluation 
collected primary outcome data that were directly 
relevant to assess intervention effects and secondary 
support data that described the supports provided to 
students and parents. 


Sample 

The study employed a two-group quasi-experimental 
design with an intervention group and a control group. 
The sample was derived from the 11 campuses that had 
participated in the Literacy Now program between 
2015-2016 and 2017-2018. During this period, 1,161 
students participated in the 25-week reading program. 
Once duplicates were identified and students connected 
to their HISD Public Education Information 
Management System (PEIMS) data, the evaluation 
sample consisted of 996 students who participated in the 
Literacy Now program. 

To measure the efficacy of the Literacy Now 
program, the evaluation used a subset sample of students 
from those campuses that participated in the program in 
2015-2016 and 2016-2017. A comparative sample of 
schools was identified using the Texas Education 
Agency Comparison Campus Group Report (2019). 
Each campus in Texas is assigned to a unique 
comparison group of schools that were most similar 
(TEA, 2019). 

TEA determined the campus comparison group by 
school type (elementary school, middle school, 


elementary /secondary, and high school) then grouped 
them with similar campuses from anywhere in Texas. 
Campuses were grouped based on similarity in grade 
levels served, size, the percentage of economically 
disadvantaged students, mobility rate, the percentage of 
English learners, the percentage of students served by 
special education, and the percentage of students 
enrolled in an early college high school program (TEA 
2019). A school could be a member of several 
comparison groups or none other than its own (TEA, 
2019). 

Comparable campuses to the Literacy Now program 
campuses were selected as the control group based on 
TEA criteria. Demographic information for the students 
from the comparison campuses were downloaded from 
the 2015-2016 and 2016-2017 PEIMS database. For 
cohort 1, the control sample consisted of 2,743 students; 
and for cohort 2, the control sample was 2,117 students. 

From the sample, a match group of students were 
selected to balance the differences in the basic 
demographic and academic characteristics between 
students who participated in the reading intervention 
program and those who did not using Propensity Score 
Matching (PSM). Propensity scores were calculated 
using logistic regression analyses based on several 
covariates; including gender, ethnicity, economically 
disadvantaged status, special education status, and 
beginning of year scale score. Students who participated 
in the reading intervention and those who did not were 
matched 1:1 based on their propensity scores using 
nearest-neighbor matching and a matching tolerance of 
0.2% for both cohorts. 

After matching the balance of all observed covariates 
were examined. Appendix A, Tables A1—A4, pp. 17- 
18, compares the means of continuous baseline 
covariates and prevalence of dichotomous baseline 
covariates between treated and untreated subjects in the 
original unmatched sample. The prevalence of 
dichotomous variables was compared between treated 
and untreated subjects using a Chi-squared test, while a 
standard two-sample t-test was used to compare 
continuous baseline covariates. 

The standardized difference was used to compare the 
mean of the continuous variable between two groups and 
to compare the prevalence of dichotomous variables 
between two groups (Austin, 2009a). For continuous 
variables, a standardized difference of less than 10% can 
indicate a negligible imbalance in a baseline covariate 
between treated and untreated subjects (Cohen, 1988). 

The Phi Coefficient was used to measure the strength 
of the association between dichotomous variables 
(Austin, 2009b). Nearly no imbalances remained for 
both cohorts as assessed through univariate and 
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multivariate tests (Appendix A, Tables A1l—A4, pp. 17— 
19). Figure 1 and Figure 2 show the actual propensity 
score distributions of both groups before and after 
matching. For cohort 1, some observations fell outside 
the area of common baseline test score (0.00; 0.20) and 
therefore were trimmed. 

The final analytical sample for cohort 1 consisted of 
388 students from 12 campuses (M-score = 195.22, S.D. = 
36.24 at pre-test). The final analytical sample for cohort 
2 consisted of 290 students from 12 campuses (M.score = 
197.41, S.D.=40.28 at pre-test). Power calculations 
indicated that with 290 students, the evaluation would 
have a 95% power to detect a difference between groups 
equivalent to Cohen’s d = 0.30 (p < 0.05, two-tailed), 
which is a small but a substantively important effect in 
education. 


Figure 1: Distribution of propensity scores of cohort 1 students who 
participated in reading intervention (“treated”) and students who did 
not (“control”) before and after matching 
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Figure 2: Distribution of propensity scores of cohort 2 students 
who participated in reading intervention (“treated”) and students 
who did not (“control”) before and after matching 
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Data Collection 

Students’ data were for more than two academic 
years (from kindergarten to third grade) downloaded and 
assessed, from a pre-intervention reading standardized 
tests (pre-test, tl) and following the 23-week 
intervention (post-test, t2). The standardized tests, 
nationally normed Istation’s Indicators of Progress for 
Reading (ISIP™ Reading), measures student growth 
using computer-adaptive diagnostic and screening 
programs for prekindergarten to eighth grade (Istation, 
2013). The objective was to identify students potentially 
at-risk of reading failure. 

The Istation assessments can be administered 
monthly, with the skill goals and criteria becoming 
progressively more difficult with each assessment period 
(station, 2013). Students’ progress was reported as 
follows: Tier 1 students performing at grade level; Tier 
2 students performing moderately below grade level and 
in need of intervention; Tier 3 students performing 
seriously below grade level and in need of intensive 
intervention (Istation, 2013). ISIP reading measures are 
highly predictive of STAAR reading scores (Mathes, 
2016; Patarapichayatham, 2017). 

The final assessment used the spring administration 
of the STAAR Grade 3 reading scale scores from 2016, 
2017, and 2018 as an academic outcome measure. The 
STAAR is an annual state-mandated criterion- 
referenced assessment that measures students’ academic 
performance and achievement. Re-testers were not 
included in the study due to overexposure to both the 
program and outcome measures. Students’ STAAR data 
used in this study were retrieved from the HISD Data 
warehouse using Cognos, a data querying platform. 

Student demographics. The demographic 
characteristics of the intervention group (n = 996) and 
control group (n = 344) used for this report were 
downloaded from PEIMS 2015-2016 to 2018-2019 
HISD student database. Characteristics included gender, 
ethnicity, economically disadvantaged status, special 
education status, English learner status, gifted/talented 
status, and home language. Each student was coded as 
‘1’ for those who participated in the Reading 
Intervention program and ‘0’ for the control group. 

Teacher survey. For students in the Reading 
Intervention program, their teacher completed a survey 
at the end of the 25-week program. There were 842 
teacher survey responses between 2015-2016 and 2017— 
2018 academic year, with a response rate of 84.5%. The 
survey included a 3-point Likert scale ranging from 
strongly agree (3) to disagree (0). The questions 
included: (i) The students exhibited greater confidence 
in reading; (ii) I felt the Making it Better, now Literacy 
Now program, was beneficial to the students, and (iii) 
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The student’s reading grades improved. Cronbach’s 
alpha (a) was used to provide evidence that the 
components of the scale were sufficiently intercorrelated 
and that the grouped items measured the underlying 
variable (Sullivan & Artino, 2013). The result of 
Cronbach's alpha was .913, denoting a rating of 
excellent. 


Data Analysis 

Data were collected for the 2015-2016 (cohort 1) and 
2016-2017 (cohort 2) students’ reading scores. 
Students’ BOY (pre-test) scores and EOY (post-test) 
scores on the Istation assessment were linked. 
Kindergarten to second grade students who had both 
scores were included in the analysis. The second part of 
the analysis further linked prekindergarten to second 
grade students’ scores on STAAR grade 3 reading 
assessment. Pre and post differences in the mean reading 
scores by grade level and demographic characteristics 
were used as measures of program impact. 


Limitations 

One limitation was the administration of student 
reading assessments. In 2018, the district changed the 
standardized evaluation that was being administered. 
The administration of 2019-2020 assessments were 
limited due to the pandemic. Therefore, students’ reading 
performance could not be traced over time. Also, there 
were various reading intervention programs that were 
being administered on HISD campuses. These other 
programs were not controlled for in this evaluation. As a 
result, there is a possibility that students in the Literacy 
Now program may have participated in other reading 
intervention programs. 

Literacy Now is a community-based program, which 
presents several challenges relating to data collection. 
First, campuses were not involved in data collection. 
Data for program activities were provided by the 
organization. Only the data presented that followed 
sound data collection protocol were included. Second, 
the Literacy Now vendor did not have access to student 
information; therefore, students that comprised the 
intervention group self-reported their IDs, general 
demographic information (first and last name), and 
campus names. The student identification information 
collected by the vendor did not correspond with the 
district PEIMS data system. Therefore, students not 
identified in the PEIMS system were excluded from the 
analysis. To mitigate the risk of including students who 
participated in the program but whose IDs were not 
available in the control sample, a sample of students 
were taken from a similar campus that did not participate 
in the program using TEA criteria. 


Results 
What was the demographic and_ educational 
composition of students and campuses that 


participated in the Literacy Now program between 
2015-2016 and 2018-2019? 


In 2007, the Literacy Now program was first 
delivered at one HISD campus, Walnut Bend 
Elementary. The Literacy Now program consists of 
several components (Reading Intervention, Parent 
Engagement Workshops, Book Distribution, Family 
Literacy Night, and monthly newsletters). 

Over the years, there have been 18 additional HISD 
campuses that participated (see Figure 3). As of the 
2019-2020 academic year, ten campuses participated in 
the Literacy Now program (see Figure 3). The average 
number of years in the program for campuses was 4.7. 
The campus that had the most years of participation in 
the program was Walnut Bend (n = 13), followed by 
Piney Point (n = 9) and Bruce (n = 9) elementary. 

Most students in the Reading Intervention program 
performed “seriously below grade level and in need of 
intensive intervention” or “moderately below grade level 
and in need of intervention” (Istation, 2013, p. 1). In 
cohort 1, of the 388 students who received the 
intervention, 68.0 percent (n = 264) of students were at 
risk of reading failure, and 15.2 percent (n=59) were 
performing moderately below grade level and in need of 
intervention (see Figure 4). Similarly, in cohort 2, of the 
290 students who received the intervention, 69.0% 
percent (n = 200) performed seriously below grade level 
and 11.4 percent (n=33) were moderately below grade 
level. 

Between 2015-2016 and 2018-2019, the average 
number of students in the Reading Intervention program 
from each campus was 91. The campuses with the 
highest number of students in the program included Red 
(12.6%, n = 125), Walnut Bend (12.6%, n = 125), Piney 
Point (11.7%. n = 117), and Bruce (10.3%, n = 103) (see 
Figure 5). 

The demographic composition of the students who 
participated in the Literacy Now Reading Intervention 
program between 2015-2016 and 2018-2019 was 
predominately male, with 52.5 percent of the population 
being male compared to 47.5 percent being female (see 
Table 1, p.8). Students in the program predominately 
were at-risk (81.7%) and economically disadvantaged 
(84.3%). There was a small percentage of students who 
were gifted/talented (1.9%) and who identified as special 
needs (3.8%). The ethnic composition of the group was 
43.7 percent Black and 43.8 percent Hispanic. 
Participants in the Reading Intervention program 
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Figure 3: HISD Campus’ years of participation in program by 
current partnership status, 2007—2020 
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Note: Those campuses with white dots participated in the program 
between 2015 and 2018, comprising the analytical sample. 


Figure 4: Students reading proficiency on Istation’s Indicators of 
Progress (ISIP) assessment by cohort 
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Figure 5: Distribution of students in reading intervention program 
by campus, 2015-2019 
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identified their home language as English (69.0%), 
followed by Spanish (21.8%). 

Analysis of student distribution by grade level 
showed that for the four cohorts who participated in the 
Literacy Now program between 2015-2016 and 2018— 
2019, students were predominately in the first grade (see 
Table 1). There were 54.1 percent of students in first 
grade 1, followed by 30.5 percent kindergarten, and 15.4 
percent were in the second grade. 

When disaggregated by years in the program, 88.6 
percent of students participated in the program for one 
year (n = 882), and 10.8 percent participated in the 
program for two years (n = 108) (see Table 2). A small 
percentage of students were in the program for 3-years 
(0.6%). Of the 108 students that participated in the 
program for two years, 53.7 percent were in kindergarten 
and 46.3 percent were in grade one. 


What Literacy Now support services were provided 
to HISD campuses from the 2015-2016 to the 2018- 
2019 academic year? 


The Literacy Now program provides several 
components (Reading Intervention, Parent Engagement 
Workshops, Book Distribution, Family Literacy Night, 
and monthly newsletters) to select HISD campuses 
between 2015-2016 to 2018-2019. For the book 
distribution, there was only campus-level data available 
for the 2016-2017 to 2018-2019 academic year. Based 
on descriptive analysis, there was a decrease in book 
distribution in 2018-2019 from the previous year (see 
Figure 6). In the 2017-2018 school year, the highest 
number of books was distributed to Walnut Bend (n = 
463), Ross (n = 458), and Shadowbriar (n=454). In 
2018-2019, the average number of books distributed 
was the same across campuses (M = 394). 

Literacy Now supported parents to encourage 
reading at home through Parent Workshops and Family 
Literacy Nights. Two Parent Workshops were offered 
each academic year: one in the fall and another in the 
spring. The highest average workshop attendance was 
from parents whose children attended Walnut Bend / 
Belmont Place Apartments (M = 20.8), Red Elementary 
(M = 26.8), Mark White Elementary (M =17.0), and 
MacGregor Elementary (M = 16.5) (see Appendix A, 
Table A5, p. 19). Participants at Belmont Place 
Apartments were parents and students who attended 
Walnut Bend Elementary. The highest attendance for 
Family Literacy Night was at Walnut Bend Elementary 
/ Belmont Place Apartments, with an average attendance 
of 106.7 parents and students (see Appendix A, Table 
AG, p. 19). Belmont Place Apartments was the only 
location with a community-after-school program. 
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Table 1: Demographic characteristics of reading intervention 
program participants, 2015-2016 through 2018-2019 


n % 
Grade Kindergarten 304 30.5 
Grade 1 539 54.1 
Grade 2 153 15.4 
Gender Female 473 47.5 
Male 523 52.5 
Ethnicity Black 435 43.7 
Hispanic 436 43.8 
Other 124 12.4 
Home Language Spanish 217 21.8 
English 687 69.0 
Other 92 9.2 
Economically No 
Disadvantaged 156 15.7 
Yes 840 84.3 
At-Risk No 182 18.3 
Yes 814 81.7 
Limited English No 717 72.0 
Proficiency Yes 279 28.0 
English Learner No 865 86.8 
Yes 131 13.2 
Gifted/ Talented No 977 98.1 
Yes 19 1.9 
Special Education No 958 96.2 
Yes 38 3.8 


Source: 2015-2019 PEIMS student databases. Percentages may 
not total 100 due to rounding. N=996 students. 


Table 2. Number of students in each grade who participated in the 
program by years in the program, 2015-2016 through 2018-2019 


Years in 


KG Grade 1 Grade 2 Total 
program 
n % n % n % n % 
1 245 80.6 484 89.8 153 100 882 886 
2 58 19.1 50 9.3 - - 108 10.8 
3 1 0.3 5 0.9 - - 6 0.6 


Figure 6: Total book distribution by campus, 2015-2016 through 
2018-2019 
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What were teachers’ observation of student’s 
progress in the Reading Intervention program from 
the 2015-2016 to the 2018-2019 academic year? 


Teachers’ mean ratings of observed improvements in 
student performance once they completed the Reading 
Intervention program is displayed in Table 3. The 
highest mean rating across grade levels was for teachers 
reporting that they felt that the intervention program was 
beneficial to their students. Teachers' observations of the 
program being beneficial had a mean rating of 2.69 for 
kindergarten students, 2.66 for second grade, and 2.53 
for first grade. 

Mean ratings for greater confidence in reading were 
higher for second-grade students (2.54), followed by 
kindergarten (2.49), and then first grade students (2.40) 
(see Table 3). Teachers agreed that there were 
improvements in the students’ grades after completing 
the intervention. There was a mean rating of 2.57 for 
kindergarten, 2.50 for second grade, and 2.47 for first 
grade students. 

There was a_ strong statistically significant 
association between teacher observation that students 
exhibited greater confidence in reading and teachers’ 
feelings that the intervention was beneficial to the 
students (r = .85) (see Table 4). There was also a 
statistically significant association between teachers 
reporting that students’ reading grades improved with 
increased confidence in reading, and that the program 
was beneficial to the students (r = .75). 

There was a weak negative (downhill sloping) linear 
relationship between being in the program for one year 
or more than one year (see Table 4). There was a weak 
negative association between years in the program with 
students exhibiting increased confidence (r = -.20), the 
program being beneficial to students (r = -.17), and 
improvement in reading grades (r = -.19). The longer a 
student participated in the program, the weaker the 
association between years of participation and teachers’ 
observations of students’ confidence, benefits, or 
reading improvements. 


How did the performance of kindergarten to second 
grade students in the Reading Intervention program 
compare with their campus grade level peers on the 
Istation reading assessment? 


A paired sample t-test was conducted comparing the 
pre-and post-academic performance for each grade level 
(kindergarten, grade 1, and grade 2) (see Appendix A, 
Table A7 for cohort 1 and Table A8 for cohort 2, p. 20). 
In both cohort 1 and cohort 2, there were significant 
differences between the pre-and post-test mean 
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Table 3: Mean rating of teacher perception of reading intervention 
program on students, teacher post-program survey responses 2015— 
2018 


N Mean S.D. 

Oh erased 254 2.49 0.80 
confidence 

Rinderganen 22: Benetclal 254 269 0.78 
students 
Q3. Reading grades 254 257 0.82 
improved 
OP ieressed 468 2.40 0.85 
confidence 

Grade 1 Ge Bene acai 469 2.53 0.80 
students 
Q3. Reading grades 470 247 0.86 
improved 
esncieased 120 254 0.78 
confidence 

Grade 2 Q2. Beneficial to 120 266 0.73 
students 
Q3. Reading grades 120 2.50 0.88 


improved 


Note: Teacher survey questions: Q1.The students exhibited greater confidence 
in reading. Q2. I felt the Making it Better Reading Intervention program was 
beneficial to the students. Q3. The student’s reading grades improved. 


Table 4: Correlation of teacher perception of student performance 
after reading intervention program, 2015-2018 


Ql Q2 Q3 Q4 
Q1. Increased confidence 1.00 
Q2. Beneficial to students 845" 1.00 
Q3. Reading grades a7" 748" 1.00 
improved 
Q4. Years in Program 195 -.172"" -.185"" 1.00 


proficiency scored in reading in the intervention group 
for kindergarten and first grade at the .01 level. In cohort 
1, kindergarten students’ pre-test results (M = 173.5, 
S.D. = 13.7) and post-test results (M = 186.3, S.D. = 
12.4) indicated that participation in the intervention 
resulted in an improvement in the mean reading 
proficiency, t(44) = 6.5, p < .001 (see Appendix A, Table 
A7, p. 20). Similarly, kindergarten students in cohort 2 
showed a significant increase in the mean reading 
proficiency score by the end of the intervention (M = 
185.2, S.D. = 13.2) compared to students’ scores prior to 
starting the intervention (M = 168.9, S.D. = 14.6), t(37) 
= 6.6, p =.000 (see Appendix A, Table A8, p. 20). The 
post-test score for both cohorts is slightly lower than the 
grade-level expected score (>200) (Istation 2013). 

For first-grade students in cohort 1, results from the 
pre-test (M = 187.6, S.D. = 10.9) and post-test (M = 
201.9, S.D. = 21.5) indicated that participation in the 
intervention resulted in an improvement in the mean 
reading proficiency, t(96) = 7.2, p =.000 (see Appendix 
A, Table A7, p. 20). Comparably, first-grade students in 


cohort 2 showed a significant increase in the mean 
reading proficiency score at the end of the intervention 
(M = 215.7, S.D. = 93.5) compared to students’ scores 
prior to starting the intervention (M = 196.7, S.D. = 
46.3), t(87) = 3.5, p = .001 (see Appendix A, Table A8, 
p. 20). The post-test score for both cohorts was slightly 
lower than the grade-level expected score (>221) 
(Istation, 2013). 

For second grade students in cohort 1, results from 
the pre-test (M = 208.7, S.D. = 11.5) and post-test (M = 
226.8, S.D. = 12.3) indicated that participation in the 
intervention resulted in an improvement in the mean 
reading proficiency, t(47) = 9.5, p = .000 (see Appendix 
A, Table A7, p. 20). However, second grade students in 
cohort 2 did not show a significant increase in the mean 
reading proficiency score at the end of the intervention 
(M = 211.5, S.D. = 21.2) compared to students’ scores 
prior to starting the intervention (M = 203.1, S.D. = 
10.2), t(18) = 1.6, p = .122 (see Appendix A, Table A8, 
p. 20). The post-test score for both cohorts was lower 
than the grade-level expected score (>238) (Istation 
2013). 

Based on these results, though not reading at grade 
level, students in both cohorts that were in kindergarten 
and first grade who participated in the reading 
intervention benefitted more from the program. To test 
this conclusion the pre- and post-means for students in 
the intervention were compared to the pre-and post- 
means for those that were not in the intervention. 

Post-hoc analysis of pair-wise group comparisons 
indicated that, across cohorts, first-grade students who 
participated in the Reading Intervention program 
performed better than their peers who did not (see 
Figure 7 and Figure 8). In cohort 1, first-grade students 
who participated in the reading intervention program 
showed a statistically significant increase in mean 
proficiency reading score (M = 14.3, S.E. = 7.2) (see 
Figure 7). While their grade-level peers who did not 
participate in the intervention also showed a significant 
increase in mean proficiency reading score (M = 13.5, 
S.E. = 4.2), it was lower than that of their peers who 
received the intervention (see Figure 7). 

For cohort 2, there was also an observed difference 
between first-grade students who participated in the 
reading intervention program and those who did not (see 
Figure 8). First-grade students who participated in the 
Reading Intervention program showed an increase in 
mean proficiency reading score (M = 19.1, S.E = 5.4) 
(see Figure 8). Their grade-level peers who did not 
participate in the program also showed a significant 
increase in mean proficiency reading score (M = 13.9, 
S.E = 1.2) (see Figure 8). However, it was lower than 
that of their peers who received the intervention. 
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Figure 7: Reading proficiency rate for cohort 1 by grade level, 


2015-2016 
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Figure 8: Reading proficiency rate for cohort 2 by grade level, 
2016-2017 
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What were the differences in reading performance 
between the students in the Reading Intervention 
program and their peers when demographic 
characteristics are taken into consideration? 


When disaggregated further by demographic 
characteristics, there was a greater increase in 
percentage points in the number of students at grade 
level on Istation’s Indicators of Progress (ISIP) 
assessment for English learners, economically 
disadvantaged, or Black students compared to their peers 
who did not receive the intervention (see Appendix A, 
Table A9 for cohort 1 and Table A10 for cohort 2, p. 
21). 

The largest increase in the number of students that 
were at grade level was among ELs for both cohorts (see 
Figure 9 and Figure 10). In cohort 1, the number of ELs 
that were at grade level increased by 56.1 percentage 
points from the pre-test (2.4%) to the post-test (58.5%) 
(see Figure 9). This was higher than the 44.5 percentage 
points increase for ELs not in the intervention, from pre- 
test (11.1%) to post-test (55.6%). 

In cohort 2, the number of ELs that were at grade 
level increased by 2.5 percentage points from the pre- 
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test (10%) to the post-test (12.5%) (see Figure 10). The 
number of EL students who did not participate in the 
intervention that were at grade level did not increase 
from the pre-test (17.6%) to the post-test (17.6%) (see 
Figure 10). 

For Black students in cohort 1, the number of 
Reading Intervention program participants that were at 
grade level increased by 42.5 percentage points from the 
pre-test (3.0%) to the post-test (45.5%) (see Figure 9). 
While the number of Black students who did not 
participate in the intervention that were at grade level 
increased by 32.3 percentage points from the pre-test 
(26.5%) to the post-test (58.8%) (see Figure 9). 

In cohort 2, the increase in reading performance 
between Black students who participated in the Reading 
Intervention program and their peers was on par. The 
number of Black students who participated in the 
Reading Intervention program that was at grade level 
increased by 2.7 percentage points from the pre-test 
(4.2%) to the post-test (6.9%) (see Figure 10). Likewise, 
the number of Black students who did not participate in 
the intervention that were at grade level increased by 2.0 
percentage points from the pre-test (31.3%) to the post- 
test (33.3%) (see Figure 10). 

The difference in the pre-test (4.3%) and post-test 
(50.0%) results for economically disadvantaged 
students, in cohort 1, showed a 45.7 percentage points 
increase in the number of students that were reading at 
grade level (see Figure 9). There was a 28.2 percentage 
points increase from pre-test (29.8%) to post-test 
(58.0%) for their peers who did not receive the 
intervention. 

For cohort 2, there was a 1.6 percentage-point 
increase in the number of economically disadvantaged 
students reading at grade level from pre-test (7.3%) to 
post-test (8.9%) compared to a 3.7 percentage points 
decrease for those who did not receive the intervention 
from the pre-test (33.8%) to the post-test (30.1%) (see 
Appendix A, Table A10, p. 21). 

Hispanic students who participated in the 
intervention did not outperform their peers in both 
cohorts, they showed a higher increase in percentage 
points in cohort 1 (see Figure 9) and no change in cohort 
2 (see Figure 10). The number of Hispanic students who 
participated in the Reading Intervention program that 
was at grade level increased by 53.6 percentage points 
from the pre-test (5.6%) to the post-test (59.2%) (see 
Figure 9). However, the number of Hispanic students 
who did not participate in the intervention that were at 
grade level increased by 23.0 percentage points from the 
pre-test (32.2%) to the post-test (55.2%) (see Figure 10). 
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Figure 9: Percentage of students at grade level on Istation’s 
Indicators of Progress (ISIP) assessment for cohort 1 by 
demographics, 2015-2016 
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Figure 10: Percentage of students at grade level on Istation’s 
Indicators of Progress (ISIP) assessment for cohort 2 by 
demographics, 2016-2017 
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What was the longitudinal effect of the Literacy Now 
program on the STAAR Reading performance of 
HISD third grade students? 


Independent sample t-tests were conducted to test the 
efficacy of the reading intervention program in helping 
to create proficient readers by the end of grade 3 (see 
Appendix A, Table Ail, p. 22). Overall, when 
examined by grade-level, the analysis found that 
students did not show significant long-term benefit from 
the reading intervention program once they got to grade 
three based on STAAR reading results. The test was 
found to be statistically significant [t(269) = -3.23, p < 
.001]. The effect size (d = .39) was found to meet 
Cohen’s (1988) convention for a small but substantively 
important effect in education. The Reading Intervention 
program had a small effect on students’ grade 3 reading 
performance. 

For cohort 1, the results indicated that the reading 
performance of third-grade students in the Reading 
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Intervention program component (M = 4662.1, S.D. = 
796.9) was lower than students who did not participate 


Figure 11: Average proficiency rate on STAAR grade 3 reading for 
cohort 1 by grade level, 2015-2016 


in the reading intervention program (M = 4979.7, S.D. = 
819.9). However, for cohort 2 there was no significant 
effect on the third-grade reading performance, t(149) = 
1.2, p = .306, despite students who participated in the 
reading intervention attaining higher grade 3 reading 
proficiency scores (M = 5032.7, S.D. = 759.2) than those 
who did not participate in the program (M = 4876.9, S.D. 
= 801.7). 

Analysis by grade level showed that of the three 
grades (kindergarten to grade 2) in cohort 1, there was a 
significant effect for kindergarten students that 
participated in the reading intervention program. The 
test was statistically significant [t(69) = -2.5, p < .015], 
and the effect size was small (d =.39) (see Appendix A, 
Table A12, p. 22). Though the difference between the 
scores for each group was significant, the third-grade 
reading performance was lower for kindergarten 
students who participated in the Reading Intervention 
program (M = 5236.7, S.D. = 622.7) than for students 
who did not participate in the reading intervention 
program (M = 5591.5, S.D. = 564.2) (see Figure 11). 

Cohort 2 included only first-grade and second-grade 
students who wrote the STAAR grade 3 reading 
assessment during the period under study (see Appendix 
A, Table A13, p. 22). Students who participated in the 
Reading Intervention program attained lower grade 3 
reading proficiency scores (M = 3859.3, S.D. = 337.7) 
than those who did not participate in the program (M = 
4212.9, S.D. = 467.1) (see Figure 12). There was not a 
significant effect on third grade reading performance for 
this cohort, [t((49) = -1.2, p= .241] (see Appendix A, 
Table A13, p. 22). 


How did students in the Reading Intervention 
program compare with their campus grade level 
peers on the STAAR grade 3 reading assessment 
when demographics are taken into consideration? 


Although analyses of the overall third grade reading 
performance found that the Reading Intervention 
program did not have a significant effect, certain grades 
and demographic groups showed moderate benefits from 
program participation. Further analyses were conducted 
to examine the difference in third grade reading 
performance by demographic characteristics (see 
Appendix A, Table A14 for cohort 1 and Table A15 for 
cohort 2, p. 22). 

The results for cohort 1 showed that for economically 
disadvantaged students, there was a_ statistically 
significant difference between the mean reading 
performance of those students who participated in the 


HISD Department of Research & Accountability 


6000 
5000 


5225.0 [5255.7 


4000 
3000 
2000 
1000 


Average Reading Proficiency 


0 


Kindergarten Grade 1 Grade 2 


Intervention m Control 


Figure 12: Average proficiency rate on STAAR grade 3 reading for 
cohort 2 by grade level, 2016-2017 
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Reading Intervention program and those who did not 
[t(240) = -3.17, p = .002] (see Appendix A, Table A14, 
p. 22). The results indicated that the reading 
performance of economically disadvantaged students in 
third grade in the Reading Intervention program (M = 
4761.2, S.D. = 786.1) was higher than students who did 
not participate in the reading intervention program (M = 
4709.6, S.D. = 790.6). 

For Black students in cohort 2, there was a 
statistically significant difference [t(240) = 3.42, p = 
.001] between the mean reading performance of those 
students who participated in the Reading Intervention 
program and those who did not (see Appendix A, Table 
A15, p. 22). The results indicated that Black students in 
the Reading Intervention program (M = 5064.9, S.D. = 
767.4) had a higher third grade reading performance than 
students who did not participate in the reading 
intervention program (M = 4434.7.6, S.D. = 727.5). The 
effect of the reading intervention program for Black 
students was large (d =.84). 

A campus-level analysis was conducted for seven of 
the ten campuses involved in the Reading Intervention 
program during the period under study (2015-2017). 
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Figure 14: Percentage of students at or above STAAR grade 3 ‘approaches’ grade level benchmark at campuses participating in Literacy 


Now program, 3-year mean (2016, 2017, 2018) 
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participated in the Reading Intervention program twice 
a week. Students at Walnut Bend Elementary 
participated in the program four times a week. 

Across cohorts, a higher percentage of students at 
Piney Point, Red, and Walnut Bend were at or above 
‘Approaches’ grade level standard on the STAAR Grade 
3 reading assessment. Piney Point Elementary had the 
highest number of students who participated in the 
reading intervention that were at or above ‘Approaches’ 
grade level standard on the STAAR reading assessment, 
with 50 percent of students in cohort 1 and 64 percent of 
students in cohort 2 (see Figure 13). 

Both cohorts at Red Elementary and Walnut Bend 
Elementary also had a high percentage of students of 
students who participated in the intervention that were at 
or above ‘Approaches’ grade level standard on the 
STAAR reading assessment. At Red Elementary, 55 
percent of students in cohort 1 and 47 percent of students 
in cohort 2 were at or above ‘Approaches’ grade level 
standard on the STAAR reading assessment (see Figure 


Figure 13: Percentage of reding intervention students that were at 
or above STAAR grade 3 ‘approaches’ grade level benchmark, 
2016, 2017, 2018 
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13). Similarly, at Walnut Bend Elementary, 52 percent 
of students in cohort 1 and 50 percent of students in 
cohort 2 were at or above ‘Approaches’ grade level 
standard on the STAAR reading assessment. 

For five of the seven campuses that participated in the 
Reading Intervention program students in the 
intervention reading performance were within 15 
percentages points of their campus peers (see Appendix 
A, Table A16, p. 23). The difference in 3-year mean 
percentage of students who met at or above 'Approaches' 
reading level benchmark on the STAAR grade 3 reading 
was within 15 percentage points of the 3-year mean for 
their campus peers at Piney Point (6%), Walnut Bend 
(10%), Ross (11%), Bruce (15%), and McGowen (15%) 
(see Figure 14). 


Discussion 


The Literacy Now program has been implemented at 
nineteen HISD schools since 2007. The program 
incorporates activities for students, parents, and the 
community that are aimed at improving student reading. 
The cornerstone of the Literacy Now program is the 
Reading Intervention program, which targets low- 
performing readers from kindergarten to second grade. 
The Reading Intervention program is tailored to meet the 
student's individual needs through a low student-to-tutor 
ratio built on partnerships with parents and the wider 
community. The goal is to create proficient readers by 
the end of third grade (Literacy Now, 2019). 

The Literacy Now early intervention program 
focused on several key elements for the implementation 
of effective reading intervention for young learners. 
These elements included (i) use of evidence-based 
curriculum and instructional materials for use in early 
intervention; (ii) scheduling time for intervention; (iii) 
selecting, training, and supporting interventionists; and 
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(iv) locating space and maintaining open 
communications among interventionists, teachers, and 
parents (Foorman, Lee, & Smith, 2020). 

The program also incorporated several key elements 
to engage parents and the community by offering Parent 
Workshops and Family Literacy Night at several 
campuses and within the community. Those campuses 
with the highest mean number of parents participating in 
the Parent Workshops included Red Elementary (M = 
26.8), Mark White (M = 17.0), and MacGregor (M = 
16.5). 

Family Literacy Night participation was high for 
Belmont Apartments, with an average attendance of 
106.7 parents and = students, and Shadowbriar 
Elementary, with an average attendance of 99 parents 
and students. Participants at Belmont Apartments were 
parents and students who attended Walnut Bend 
elementary. At these events, students and parents 
received books to support reading at home. 

However, there was a lack of cohesiveness across 
data collection for the various components of the 
program and student performance, making it challenging 
to measure the program's effectiveness based on the 
level of participation in each activity. This lack of 
cohesiveness across program components limited the 
capacity of the program to ensure adherence to the level 
of fidelity meeded for successful program 
implementation. 

As pointed out by Foorman, Lee, and Smith (2020), 
these additional elements (parent support, small group 
size, evidence-based curriculum) are important aspects 
that are critical to the successful implementation of early 
literacy intervention. It is necessary to ensure that 
students have received research-based interventions 
implemented with high levels of fidelity to assess the 
effectiveness of the intervention (Austin, Vaughn, & 
McClelland, 2017). 

In terms of the Reading Intervention program, data 
were collected from the teachers of students who 
participated in the program. The results of the post- 
program survey administered to teachers revealed that 
instructors observed that students displayed greater 
confidence in reading after participation in the program. 
Across grade levels, teachers reported that the program 
was beneficial for students. The highest mean rating, on 
a 3.0 scale, was for kindergarten students (M = 2.69), 
followed by second grade (M = 2.66), and first grade (M 
= 2.53). 

The evaluation found that for both cohorts of 
kindergarten and grade 1 students who participated in 
the Reading Intervention program, there was a 
significant difference between the pre-and post-test 
scores for the average rate of proficiency in reading. 


HISD Department of Research & Accountability 


However, when compared to students who did not 
participate in the Reading Intervention program, 
students in grade 1 showed a higher mean difference 
across cohorts. 

The evaluation found that certain demographic 
groups responded better to the reading intervention. The 
pre-test and post-test results for students who were 
English learners, economically disadvantaged, or Black 
indicated that participation in the intervention resulted in 
an improvement in the mean reading proficiency. The 
largest average increase was among EL students for both 
cohorts. Pull-out interventions have been found to have 
transformative effects on students’ reading performance, 
especially in high-poverty schools and among English 
learners (Everson & Bussey, 2013). 

A campus-level analysis was conducted to examine 
the third-grade reading performance of students who 
participated in the Reading Intervention program. It was 
found that campuses involved in the Literacy Now 
program, apart from Piney Point, had fewer students 
who performed at or above the ‘Approaches’ grade level 
standard on STAAR grade 3 reading assessment 
compared to their campus-level peers. 

There was a fifteen mean percentage points 
difference between the number of students who were at 
or above the ‘Approaches’ grade level standard on 
STAAR grade 3 reading assessment compared to their 
campus-level peers at several campuses. These 
campuses included: Piney Point (6%), Walnut Bend 
(10%), Ross (11%), Bruce (15%), and McGowen (15%). 

Students who participated in the Literacy Now 
Reading Intervention program have shown pre-and post- 
test improvements. A significant goal of an intervention 
is to accelerate student progress to reduce the gap 
between students’ current reading level and grade-level 
expectations (Vaughn, Denton, & Fletcher, 2010). 
However, caution should be taken when interpreting the 
findings of this evaluation since it was not feasible to 
identify the interaction between the components of the 
Literacy Now program (Reading Intervention, Parent 
Engagement Workshops, Book Distribution, Family 
Literacy Night, and monthly newsletters) offered to 
HISD students and the fidelity of the intervention to 
accelerate reading performance. 

Since the period under study, Literacy Now has made 
several changes to the program. One of which is the 
integration of the Collaborative Classroom SIPPS 
(Systematic Instruction in Phonological Awareness, 
Phonics, and Sight Words) curriculum with the Neuhaus 
Reading Readiness program. SIPPS is a research-based 
foundational skills program proven to help both new and 
struggling readers build skills and confidence for fluent 
and independent reading (Literacy Now, 2020). Literacy 
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Now Interventionists are trained on SIPPS and Neuhaus 
Reading Readiness. Additionally, Literacy Now has 
started the process of implementing some of the 
recommendations from this evaluation relating to data 
collection, assessment, and fidelity. Therefore, it would 
be beneficial to further examine the impact of the 
program on the long-term reading performance of HISD 
kindergarten to grade three students. 
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APPENDIX A 


Table A1. Comparison of baseline characteristics between treated and untreated subjects in the original unmatched sample, 2015-2016 


non-Participant (N = 2,743) Participant (N = 199) Standardized 
= = p-value diff 
n % n % ifferences 
BOY Score 376.68 205.72 
2,743 (250.89) 199 (90.91) 0.00 0.69 
Grade Kindergarten 889 32.4 49 24.6 0.00 0.08 
Grade 1 946 34.5 99 49.7 
Grade 2 938 34.2 51 25.6 
Gender Female 1,369 49.9 90 45.2 0.12 0.02 
Male 1,374 50.1 109 54.8 
Ethnicity Black 710 25.9 104 52.3 0.00 0.16 
Hispanic 1,838 67.0 76 38.2 
Other 195 71 19 9.5 
Home Language Spanish (1) 1,237 45.1 36 18.1 0.00 0.14 
English (98) 1,375 50.1 146 73.4 
Other (99) 131 4.8 17 8.5 
Economically No 466 17.0 29 14.6 0.22 0.02 
Disadvantaged Yes 2,277 83.0 170 85.4 
At-Risk No 796 29.0 21 10.6 0.00 0.10 
Yes 1,947 71.0 178 89.4 
EL No 1,431 52.2 153 76.9 0.00 -0.12 
Yes 1,311 47.8 46 23.1 
Gifted/ Talented No 2,579 94.0 196 98.5 0.00 -0.05 
Yes 164 6.0 3 1.5 
SPED No 2,579 94.0 189 95.0 0.28 0.01 
Yes 164 6.0 10 5.0 


Source: 2015-2019 PEIMS student databases. Percentages may not total 100 due to rounding. 
Notes: Continuous variables are reported as mean + standard deviation. Dichotomous variables are reported as N (Per cent). 
Phi Coefficient: 0 is no relationship; 1 is a perfect positive relationship; -1 is a perfect negative relationship. 


Table A2. Comparison of baseline characteristics between treated and untreated subjects in the original matched sample, 2015-2016 


os a Standardized 
non-Participant (N = 2,743) Participant (N = 199) p-value cram anes: 
n % n % 
BOY Score 205.72 
199 201.72 (49.65) 199 (90.91) 0.08 -0.05 
Grade Kindergarten 66 33.2 49 24.6 0.06 0.12 
Grade 1 77 38.7 99 49.7 
Grade 2 56 28.1 51 25.6 
Gender Female 91 45.7 90 45.2 0.50 0.01 
Male 108 54.3 109 54.8 
Ethnicity Black 102 51.3 104 52.3 0.06 0.12 
Hispanic 89 44.7 76 38.2 
Other 8 4.0 19 9.5 
Home Language Spanish (1) - 0.0 36 18.1 0.00 0.32 
English (98) 181 91.0 146 73.4 
Other (99) 18 9.0 17 8.5 
Economically No 9 4.5 29 14.6 0.00 -0.17 
Disadvantaged Yes 190 95.5 170 85.4 
At-Risk No 103 51.8 21 10.6 0.00 0.45 
Yes 96 48.2 178 89.4 
EL No 181 91.0 153 76.9 0.00 0.19 
Yes 18 9.0 46 23.1 
Gifted/ Talented No 198 99.5 196 98.5 0.31 0.05 
Yes 1 0.5 3 1.5 
SPED No 194 97.5 189 95.0 0.15 0.07 
Yes 5 2.5 10 5.0 


Source: 2015-2019 PEIMS student databases. Percentages may not total 100 due to rounding. 
Notes: Continuous variables are reported as mean + standard deviation. Dichotomous variables are reported as N (Per cent). 
Phi Coefficient: 0 is no relationship; 1 is a perfect positive relationship; -1 is a perfect negative relationship. 
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Table A3. Comparison of baseline characteristics between treated and untreated subjects in the original unmatched sample, 2016-2017 


ree = Participant (N = Standardized 
non-Participant (N = 2,117) 145) p-value difieverices 
n % n % 
BOY Score 392.71 190.23 
2,117 (249.37) 145 (39.09) 0.00 0.82 
Grade Kindergarten 697 32.9 38 26.2 0.00 0.16 
Grade 1 671 31.7 88 60.7 
Grade 2 749 35.4 19 13.1 
Gender Female 1,065 50.3 68 46.9 0.24 -0.02 
Male 1,052 49.7 77 53.1 
Ethnicity Black 541 25.6 72 49.7 0.00 0.14 
Hispanic 1,456 68.8 60 41.4 
Other 120 5.7 13 9.0 
Home Language Spanish (1) 977 46.2 30 20.7 0.00 0.14 
English (98) 1,052 49.7 100 69.0 
Other (99) 88 4.2 15 10.3 
Economically No 258 12.2 22 15.2 0.18 -0.02 
Disadvantaged Yes 1,859 87.8 123 84.8 
At-Risk No 446 21.1 19 13.1 0.01 0.05 
Yes 1,671 78.9 126 86.9 
EL No 1,086 51.4 105 72.4 0.00 -0.10 
Yes 1,028 48.6 40 27.6 
Gifted/ Talented No 2,049 96.8 140 96.6 0.51 0.00 
Yes 68 3.2 5 3.4 
SPED No 2,050 96.8 138 95.2 0.33 0.02 
Yes 67 3.2 7 4.8 


Source: 2015-2019 PEIMS student databases. Percentages may not total 100 due to rounding. 
Notes: Continuous variables are reported as mean + standard deviation. Dichotomous variables are reported as N (Per cent). 
Phi Coefficient: 0 is no relationship; 1 is a perfect positive relationship; -1 is a perfect negative relationship. 


Table A4. Comparison of baseline characteristics between treated and untreated subjects in the original matched sample, 2016-2017 


ee ex Standardized 
non-Participant (N = 145) Participant (N = 145) p-value Ai Previa nae 
n % n % 
BOY Score 190.23 
145 204.60 (40.30) 145 (39.09) 0.07 0.36 
Grade Kindergarten 32 22.1 38 26.2 0.00 0.26 
Grade 1 62 42.8 88 60.7 
Grade 2 51 35.2 19 13.1 
Gender Female 67 46.2 68 46.9 0.50 0.01 
Male 78 53.8 77 53.1 
Ethnicity Black 48 33.1 72 49.7 0.02 0.17 
Hispanic 80 55.2 60 41.4 
Other 17 11.7 13 9.0 
Home Language Spanish (1) - 0.0 30 20.7 0.00 0.34 
English (98) 126 86.9 100 69.0 
Other (99) 19 13.1 15 10.3 
Economically No 12 8.3 22 15.2 0.05 -0.11 
Disadvantaged Yes 133 91.7 123 84.8 
At-Risk No 49 33.8 19 13.1 0.00 0.24 
Yes 96 66.2 126 86.9 
EL No 127 88.2 105 72.4 0.00 0.20 
Yes 17 11.8 40 27.6 
Gifted/ Talented No 143 98.6 140 96.6 0.22 0.07 
Yes 2 1.4 5 3.4 
SPED No 140 96.6 138 95.2 0.39 0.04 
Yes 5 3.4 7 4.8 


Source: 2015-2019 PEIMS student databases. Percentages may not total 100 due to rounding. 
Notes: Continuous variables are reported as mean + standard deviation. Dichotomous variables are reported as N (Per cent). 
Phi Coefficient: 0 is no relationship; 1 is a perfect positive relationship; -1 is a perfect negative relationship. 
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Table A5: Number of participants in Literacy Now Parent Workshops by campus, 2015-2019 


2015-16 2016-17 2017-18 2018-19 Mean 
Ashford 16 16 18 14 16.0 
Bruce 6 6 7 4 5.8 
MacGregor - - 15 18 16.5 
Mark White - - 19 15 17.0 
McGowen 2 2 - - 2.0 
Piney Point 9 9 11 10 9.8 
Red 29 29 20 29 26.8 
Ross 7 7 8 17 9.8 
Shadowbriar - - 15 14 14.5 
Shearn 3 3 - - 3.0 
Walnut Bend 5 5 13 19 10.5 
Belmont Apartments 5 10 16 10 10.3 


Note: Data provided by the Literacy Now Program; '-' indicates no data were provided for that 


year / campus. 


Table A6: Number of participants in Literacy Now Family Literacy Night by campus, 2015-2019 


2015-16 2016-17 


Ashford - - 
Bruce - - 
MacGregor - - 
Mark White - - 
McGowen - - 
Piney Point 82 - 
Red - - 
Ross - - 
Shadowbriar 

Shearn - - 
Walnut Bend - - 
Belmont Apartments 113 - 


86 


136 


2017-18 


2018-19 


71 


Mean 


106.7 


Note: Data provided by the Literacy Now Program; '-' indicates no data were provided for that 


year / campus. 
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Table A7: Paired sample t-test results for cohort 1 pre- and post-intervention test results, 2015-2016 


Intervention Control 
Paired t-test 
mean 95% t- mean 95% t- 
Mean De sa Ga. Wate 1 - MiaD Pe diene. Ga. Sean a. 
- (n = 64) 
Kindergarten (re) 
173.5 13.7. 12.8 Oe: 6.5 44 0.00 186.9 64.2 20.3 199 9.27 63 0.000 
tl : , : 16.8 : : : , : 24.6 : : 
t2 186.3 12.4 207.1 73.0 
Grade 1 (n = 97) (n = 76) 
187.6 10.9 14.3 10-4, 7.2 96 0.00 198.6 20.8 13.5 ee 10.87 75 0.000 
tl : , : 18.3 : , : , ‘ 15.9 : : 
t2 201.9 21.5 212.0 21.5 
Grade 2 (n = 49) (n = 56) 
208.7 11.5 18.1 eee 9.5 47 0.00 213.4 19.0 8.6 oe 6.63 55 0.000 
tl i : : 21.9 : ; ; ; : 11.2 é : 
t2 226.8 12.3 222.0 20.2 


Source: 2015-2016, 2016-2017, and 2018-2019 STAAR database and HISD District and School Profiles publication 


Table A8: Paired sample t test results for cohort 2 pre- and post-academic performance results, 2016-2017 


Intervention Control 
Paired t-test 
mean 95% t- mean 95% t- 
Pe le ait: ied _walnee he Mead: 2208 <ait Jets sane 
Kindergarten (n = 38) (n = 31) 

1689 146 163 ‘' 66 37 0.000 199.3 186 -24 “2 9.2 30 0.85 
t1 ; : : 21.3 : : : : : 23.0 : , 
t2 185.2 13.2 196.9 72.8 

Grade 1 (n = 88) (n = 63) 

196.7 46.3 19.1 BS, 3.5 87 0.001 196.9 17.9 13.9 a 11.9 0.00 
t1 : : : 29.8 : ; : : ‘ 16.2 : 61 ~ 
t2 215.7 93.5 210.8 17.4 

Grade 2 (n = 19) (n= 51) 

203.1 10.2 8.4 Be 1.6 18 0.12 219.9 19.7 9.2 O78, 7.4 50 0.00 
t1 : : : 19.3 , , : ; : 11.8 : , 
t2 211.5 21.2 229.1 20.8 


Source: 2015-2016, 2016-2017, and 2018-2019 STAAR database and HISD District and School Profiles publication 
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Table A9: Students’ performance tier on Istation’s Indicators of Progress (ISIP) assessment for cohort 1 by demographics, 2015-2016 


tl (pre-test) 


Intervention 


t2 (post-test) 


Percentage 


tl (pre-test) 


Control 


(2 (post-test) 


Percentage 


ISIP Tier n % n % points change n % n % points change 
English Learner (ELs) 
At grade level 1 2.4 24 58.5 56.1 2 111 10 55.6 44.5 
Moderately below grade level 2 4.9 5 12.2 7.3 5 27.8 1 5.6 -22.2 
Seriously below grade level 38 92.7 12 29.3 -63.4 11 61.1 7 38.9 -22.2 
Economic Disadvantaged 
At grade level 7.0 4.3 81.0 50.0 45.7 56 29.8 109 58.0 28.2 
Moderately below grade level 16.0 9.9 33.0 20.4 10.5 36 19.1 35 18.6 -0.5 
Seriously below grade level 139.0 858 480 29.6 -56.2 96 51.1 44 23.4 -27.7 
Black 
At grade level 3.0 3.0 46.0 45.5 42.5 27 26.5 60 58.8 32.3 
Moderately below grade level 12.0 11.9 15.0 14.9 3.0 16 15.7. 14 = 13.7 -2.0 
Seriously below grade level 86.0 85.1 40.0 39.6 -45.5 59 57.8 28 27.5 -30.3 
Hispanic 
At grade level 4.0 5.6 42.0 59.2 53.6 28 32.2 48 55.2 23.0 
Moderately below grade level 8.0 11.3 17.0 23.9 12.6 22 25.3 22 25.3 0.0 
Seriously below grade level 59.0 83.1 12.0 16.9 -66.2 37 42.5 1719.5 -23.0 


Note: Bolded numbers are the at or above 'Approaches' Grade Level Standard on STAAR Grade 3 Reading discussed in the report. 


Table A10: Students’ performance tier on Istation’s Indicators of Progress (ISIP) assessment for cohort 2 by demographics, 2016-2017 


Intervention Control 
t1 (pre-test) t2 (post-test) Percentage tl (pre-test) _t2 (post-test) Percentage 

ISIP Tier n % n % points change n % n % points change 
English Leamer (ELs) 

At grade level 4 10.0 5 12.5 2.5 3 17.6 3 17.6 0.0 

Moderately below grade level 4 10.0 7 17.5 7.9 1 5.9 1 5.9 0.0 

Seriously below grade level 32 80.0 28 70.0 -10.0 13 76.5 13 76.5 0.0 
Economic Disadvantaged 

At grade level 9.0 7.3 11.0 8.9 1.6 45 33.8 40 30.1 -3.7 

Moderately below grade level _—8.0 6.5 21.0 17.1 10.6 20 15.0 35 26.3 11.3 

Seriously below grade level 106.0 86.2 91.0 74.0 -12.2 68 51.1 58 43.6 -7.5 
Black 

At grade level 3.0 4.2 5.0 6.9 2.7 15 31.3 16 33.3 2.0 

Moderately below grade level 4.0 5.6 13.0 18.1 12.5 10 20.8 12 25.0 4.2 

Seriously below grade level 65.0 90.3 540 75.0 -15.3 23 47.9 20 41.7 -6.2 
Hispanic 

At grade level 6.0 10.0 6.0 10.0 0.0 30 37.5 28 35.0 -2.5 

Moderately below grade level 4.0 6.7 11.0 18.3 11.6 14 17.5 25 31.3 13.8 

Seriously below grade level 50.0 83.3. 43.0 71.7 -11.6 36 45.0 27 33.8 -11.2 


Note: Bolded numbers are the at or above 'Approaches' Grade Level Standard on STAAR Grade 3 Reading discussed in the report. 
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Table A11: Independent sample t test results for cohort 1 and cohort 2 grade 3 STAAR English Reading, 2015-2016 and 
2016-2017 


Intervention Control 
N M S.D. N M S.D. t-value df D 
Cohort 1 130 4662.1 796.3 141 4979.7 819.9 -3.23 269 ~=0.001 
Cohort 2 63 5032.7 759.2 88 4876.9 801.7 1.204 149 ~=0.306 


Table A12: Independent sample t test results for cohort 1 grade 3 STAAR English Reading by grade level, 2015-2016 


Intervention Control 
N M S.D. N M S.D. t-value df D 
Kindergarten 30 5236.7 622.7 41 5591.5 564.2 -2.505 69 0.015 
Grade 1 65 4093.8 563.3 53 4261.5 449.8 -1.757 116 0.081 
Grade 2 35 5225.0 493.0 47 5255.7 731.2 -0.215 80 0.830 


Table A13: Independent sample t test results for cohort 2 grade 3 STAAR English Reading by grade level, 2016-2017 


Intervention Control 
N M S.D. N M S.D. t-value df D 
Kindergarten . = : = : 2 = 
Grade 1 49 5367.9 444.2 46 5483.0 506.6 -1.179 93 0.241 
Grade 2 14 3859.3 337.7 42 4212.9 467.1 -2.608 54 0.012 


Table A14: Independent sample t test results for cohort 1 grade 3 STAAR English Reading by demographics, 2015-2016 


Intervention Control 
N M S.D. N M S.D. t-value df D 
English Learner 30 4651.8 809.3 16 4792.7 979.5 -0.522 44 0.604 
Econ. Dis. 109 4640.3 814.1 133 4973.5 812.1 -3.172 240 0.002 
Black 67 4761.2 786.1 77 4709.7 790.6 0.391 142 0.696 
Hispanic 48 4511.6 789.0 59 5292.1 703.9 -5.403 105 0.000 


Table A15: Independent sample t test results for cohort 2 grade 3 STAAR English Reading by demographics, 2016-2017 


Intervention Control 
N M S.D. N M S.D. t-value df D 
English Learner 15 4884.0 713.4 10 4912.7 707.7 -0.099 23 0.922 
Econ. Dis. 55 5039.3 750.7 81 4894.8 815.0 1.048 134 0.297 
Black 30 5064.9 767.4 36 4434.7 727.5 3.418 64 0.001 
Hispanic 28 5026.1 712.1 42 5183.5 723.4 -0.897 68 0.373 
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Table A16: Percentage of students that attained ‘Approaches’ on STAAR grade 3 reading, 2016-2018 


Intervention Overall Campus 
Cohort 1 Cohort 2 3-year 2016 2017 2018 3-year Mean 
n % n % Mean % % % % % 
Ashford 8 38 4 25 31 48 50 50 49 
Bruce 16 38 - - 38 57 49 54 53 
McGowen 20 35 4 25 30 30 48 58 45 
Piney Point 18 50 11 64 57 57 65 67 63 
Red 20 55 15 47 51 75 82 79 79 
Ross 24 38 17 47 42 46 60 53 53 
Shadowbriar - - 2 0 0 60 62 59 60 
Shearn 2 0 - - 0 47 59 58 55 
Walnut Bend 23 52 10 50 51 69 63 50 61 
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